Optimal Unravellings for Feedback Control in Linear Quantum Systems 
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For quantum systems with hnear dynamics in phase space much of classical feedback control theory 
applies. However, there are some questions that are sensible only for the quantum case, such as: 
given a fixed interaction between the system and the environment what is the optimal measurement 
on the environment for a particular control problem? We show that for a broad class of optimal 
(state-based) control problems (the stationary Linear- Quadratic- Gaussian class), this question is a 
semi-definite program. Moreover, the answer also applies to Markovian (current-based) feedback. 
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Classical feedback control is a large and well estab- 
lished area of engineering P, 0] . Quantum feedback con- 
trol, by contrast, is still rapidly developing (see Ref. Q 
for a recent review of theory) especially with the notable 
success of recent experiments in the field 0, 0, IE ■ Al- 
though problems in quantum control arc generally quite 
distinct from their classical counterparts, for quantum 
systems with linear dynamics in phase space, and with 
Gaussian noise, classical optimal feedback control theory 
can be applied. This idea was introduced by Belavkin |S| , 
and its physical application was developed in Refs. 0- In 
linear systems the key differences between the quantum 
and classical problems are the limits resulting from mea- 
surement backaction noise. The significance of this field 
is that in many quantum feedback control experiments a 
linear theory can be used, for example in quantum optics 
0, 01 , atomic ensembles 0, 0| , nanomechanics 0| 
and cavity QED 0. 

This Letter contains several important advances in this 
field. First, we give a completely general theory of feed- 
back control in linear quantum systems with a fixed cou- 
pling to the environment, allowing for arbitrary unrav- 
ellings (ways of monitoring the environment). Second, 
having formulated the conditional evolution under ar- 
bitrary unravellings as stochastic differential equations 
for the moments, we show that there is a stabilizing so- 
lution W for the conditioned covariance matrix under 
weaker conditions than classically. Third, we identify 
which unconditional evolutions allow for such a, W to 
exist for almost all unravellings, and show that the set 
of all possible Ws is set by two linear matrix inequalities 
(LMIs) . Fourth, wc show that for all stationary problems 
with a cost function that is quadratic in the system and 
controller variables, the optimal unravelling for optimal 
feedback control can be found efficiently using a semi- 
definite program. Fifth, we show that if all control con- 
straints are removed, Markovian feedback control (which 
is much simpler both conceptually and experimentally) 
performs identically to optimal feedback control. We con- 
clude with an example followed by discussion. 



Continuous Markovian Unravellings. — The most gen- 
eral autonomous differential equation for the state of a 
quantum system is the Lindblad master equation 



hp= -i[H,p]+J2'^i^i]P = ^' 



oP 



(1) 
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Here H = W is the system Hamiltonian, {c;} is a set 
of operators that are arbitrary (although for rigour [l^ 
lisj they should be bounded). The action of 2?[c] on an 
arbitrary operator p is defined by 



I?[c]p = cpc^ — (^c'cp + pc^c) /2. 



(2) 



Lindblad master equations are widely used in atomic, op- 
tical, and nuclear physics 0|. They can typically be de- 
rived if the system is coupled weakly to an environment 
that is large (i.e. with dense energy levels). Because 
the system typically becomes entangled with the envi- 
ronment, tracing over the latter typically leads to loss 
of system purity. However, there is no necessity to ig- 
nore the environment. Indeed, under the conditions that 
allow the derivation of a Lindblad master equation, it 
is possible to measure the environment continually on a 
time scale much shorter than any system time of inter- 
est. This monitoring yields information about the sys- 
tem, producing a stochastic conditioned system state pc 
that on average reproduces the unconditioned state p. 
That is, the master equation is unravelled into stochastic 
quantum trajectories [l^ . with different measurements 
on the environment leading to different unravellings. 

If we ask that the monitoring also yield an evolution for 
Pc that is, like that of Eq. ffl, continuous and Markovian, 
then it must be of the form [ijj 

hdpc = CoPcdt + dz\t)AcCpc + Pc^cC' dz(t). (3) 

Note that here the f indicates transpose (T) of the 
vector and Hermitian adjoint of its components, and 
c = (ci, • • • , cl)^. We are also using the notation 
AcO = 6 — (o)j., where (6)^ = Tr[pco], and we have in- 
troduced dz = [dzi, • • • , dzL)^ of infinitesimal complex 
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Wiener increments |20j. It satisfies E[dz] = 0, where E 
denotes expectation value, and has the correlations 



dzdz^ = hOdt, dzdz^ = hTdt. 



(4) 



Here we have generalized Ref. 0| to allow for inefficient 
detection by introducing a matrix Q. It is convenient to 
combine T and O in an unravelling matrix 



e + Re [T] Im [T] 
Im [T] e - Re [T] 



(5) 



The set it of vahd Us is defined by C/ > 0, T = T^, 
and 9 = diag(6'i, ■ ■ ■ ,6l), where 9i G [0, 1]. This can be 
derived using the method of Ref. 0| by splitting each 
output channel / into a portion 6i that is observed and a 
portion 1 — 0; that is unobserved. In the case of efficient 
monitoring (O = /), Eq. preserves purity and so can 
be replaced by a stochastic Schrodinger equation 

The measurement results upon which the evolution of 
Pc is conditioned is a vector of complex functions 



J^dt : 



(6) 



In optics measurements like this (e.g. homodync detec- 
tion) give rise to photocurrents, so we call J a current. 

Linear Systems. — We are interested in systems of N 
degrees of freedoms, with the nth described by the canon- 
ically conjugate pair obeying the commutation relations 
[^niPn] = if^- Defining a vector of operators 



{qi,Pi, ...,qN,PNV > 



(7) 



we can write [a;„,a;m] = ihY,nm, where E is the {2N) x 
(2A^) symplectic matrix 



N 



1 




(8) 



For a system with such a phase-space structure it is 
possible to define a Gaussian state. Like its classical 
counterparts, it is determined by its mean vector (x) and 
its covariance matrix, which in the quantum case must 
be symmetrized: Km = ((Ai„ Ax„) -f (AxmAi„)) /2. 
For such states, a necessary and sufficient condition on 
the matrix V is the following LMI 



V + ihS/2 > 0. 



(9) 



In this Letter we are concerned with linear systems; 
that is, ones for which H is quadratic, and c linear, in x: 



= (l/2)i^Gi-i^SSu(t), c = Ci, 



(10) 



where G is real and symmetric and B is real. The second 
term in H is linear in x to ensure a linear map between 
the time-dependent classical input u{t) to the system and 



the output current J(t). For such a system, the uncon- 
ditioned master equation |^ has a Gaussian state as its 
solution, with the following moment equations 



d{x)/dt = A(x) +Bu{t), 
dV/dt ^ AV + VA^ + D. 



(11) 
(12) 



Here A = S(G + Im[C'tc']) and D = hJ:Re[C^ C]^'^ . 

For conditional evolution of linear quantum systems it 
is convenient to recast the complex current J of Eq. (jSj) 
as a real current with uncorrelated noises: 



(hU) 



-1/2 



Re [J] 
Im [J] 



(13) 



Here C = 2{U/h)^/^C, where = (Re[C^],liTi[C 
while dw is a vector of real Wiener increments satisfying 
dwdw^ = Idt The state pc conditioned on y{t) 

is found from Eq. For linear systems this condi- 

tioned state is Gaussian, and is analogous to the a poste- 
riori probability distribution propagated by the classical 
Kalman filter equations, with dw as the innovation 

Taking expectation values and using the Ito calculus 
[20| . Eq. ^ yields the conditional moment equations 

d(x)c = [A(x)c + Bu{t)] dt + (KC^ + r^) dw (14) 
Vc - AV^ + VcA^ + D- {VcC^ + T^){CVc + F), (15) 
where F = -{hUy^^ SCJ:'^ , where 

/ 



s = 



-I 



(16) 



The stochastic term in d(x)c and the final term in 14 
describe the conditioning on the measured current y. 

Note that the equation for Vc is deterministic and in- 
dependent of the measurement results. The final term 
causes a reduction in uncertainty about the system state 
(that is, a reduction in the eigenvalues of 14). If Eq. H15|) 
has a stabilizing solution 0, then all initial conditions 
will asymptote to the same steady state. We will no- 
tate a stabilizing solution V^'^'^ as Wu to emphasize that 
it depends upon the unraveling U. Considering efficient 
detection for simplicity, Wu is the solution of 

O^nWu + Wu^^ ~WuC^CWu + EE^. (17) 

Here fl = S[G + C'^S'(2;7 - I)C] is Hamiltonian drift, 
while E — /iEC^/2, manifesting the measurement back- 
action noise resulting from having y cx Cx. For a general 
classical problem, fl, C, and E would be unrelated. 

A necessary condition for Eq. (|17|l to have a stabilizing 
solution is for (C, fl) to be detectable 0, which means 



Cv 7^ Vv : r^v = Av with Rc(A) > 0. 



(18) 



That is, all the degrees of freedom that are not strictly 
stable under the drift matrix Q contribute to the sig- 
nal Cx. Classically this becomes a sufficient condition 
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if we have also {E^ detectable [3- Quantally. this 
extra assumption is unnecessary because of back-action. 
To see this, note that from Eq. (fTH|l . (C, fi) detectable 
=> (CE^,rJE^) detectable by the invertibihty of E; 
then by the symmetry of fiS^ this implies (CE^,Er2^) 
detectable, which impHes fi^) detectable as desired. 
That is, unlike the classical case, detectability of (C, A) 
[which is equivalent to detectability of (C, il)] is sufficient 
for Wu to be a stabilizing solution of Eq. l|T^ . 

Possible Conditional States. — Let the U ~ 1/2 
unravelling be detectable [i.e. let (C, A) be de- 
tectable]. Then the Wjjs that result from detectable 
unravcUings can be shown to be dense in the set 
W = {Wu : U e !d,Wu < 00} of finite solutions Wu to 
Eq. ((nil- Moreover if the U = 1/2 unravelling is not 
detectable then no unravelling is, because Cv = =^ 
Cv = 0. Thus, for all practical purposes, any Wu e 2IT 
will be a stabilizing solution if and only if (C, A) is de- 
tectable (an unravelling-independent condition). 

The set W can be determined as follows. Note that 
from the positivity of the final term in Eq. H15() , 

D + AWu + WuA^ > (19) 

is a necessary condition on Wu- This condition means 
that a state with Vc = Wu will evolve unconditionally to 
a state with V = Wu + dt{D + AWu + WuA^) > Wu, 
in other words to a convex (Gaussian) combination of 
states with V = Wu- By obtaining information from the 
bath, the system can thus be steered back to a state with 
Vc = Wu UM- That is, Eq. fH^ is also a sufhcient condi- 
tion on Wu for the existence of some U which will yield 
V^^ = Wu, provided that Wu is a valid covariance ma- 
trix {Wu > classically, and Wu +1^^1/2 > quantally). 
Moreover, given a Wu that satisfies the LMIs (fT^ and © 
(with V Wu), a (not necessarily unique) unravelling 
U that will generate it can be found by solving 

hR^UR^ D + AWu + WuA^ , (20) 

where R = 2CWu /fi + SCT,. This comes from substitut- 
ing the expressions for O, C and E into Eq. (|17|l . 

Optimal Quantum Control. — In feedback control, u(t) 
depends on the history of the measurement record y(s) 
for s < t. The typical aim of control over some interval 
[ioj^il is to minimize the expected value of a cost func- 
tion [l|, the integral of the sum of positive functions of 
x(f) and u(t) for to <t <ti. For any such problem the 
separation principle holds: the optimal control u(t) de- 
pends only upon the observer's state of knowledge about 
the system. That is, in the quantum case, the measure- 
ment record y(s) for t < s is irrelevant except in how 
it determines Pc{t). A special case of interest is that of 
Linear-Quadratic-Gaussian (LQG) control Q]: a linear 
system with a quadratic cost function and having Gaus- 
sian noise. LQG control has the additional property of 
certainty equivalence: the only property of pc required is 



(x)^. Moreover, the optimal u is linear in this mean: 

u{t)^-K{t){i)^{t). (21) 

In this Letter we specialize to the case of time- 
independent cost functions, and (<i — <o) ~* 00. That 
is, we wish to minimize m = E[/i] in steady state, where 

= (x^Px)^ + u^Qu, (22) 

where P > and Q > 0. Note that in steady state 

E,,[(xTpx)J =tr[I^yP] +E3.[(x)JP(x)J, (23) 

just as in classical control. Assuming that Wu is sta- 
bilizing, Q~^B exists, and {B^ ,AJ) and {P,AJ) are 
detectable, there is a stable optimal control law: K = 
Q^^B^Y Here Y is independent of U, satisfying 
P + A^Y + YA = YBQ-^B^Y. The resulting (mini- 
mum) cost is Q 

mopt = tv[YBQ-^B'^YWu] + tv[YD]. (24) 

The significance of Eq. (|24|1 is that it makes it explicit 
that the choice of unravelling U (which determines Wu) 
affects the cost of the control. Thus we can ask the follow- 
ing: given an open system with dynamics described by 
the drift A and diffusion D matrices, what is the optimal 
way to monitor the bath? Note that classically this would 
be a nonsensical question as the unconditioned evolution 
described by A and D would not proscribe the measure- 
ments that can be made on the system in any way. But 
for quantum systems Wu will be positive definite be- 
cause there is no measurement without disturbance, and 
so mopt 7^ even for zero control cost (Q — > 0). 

Finding the optimum unravelling ([/) is computation- 
ally efficient in the system size N . This is because at its 
heart is the semi-definite program |23l | of minimizing a 
linear function of Wu (j24|l subject to the constraints of 
the LMIs © (with V Wu) and The optimum 

unravelling U is then found from Eq. H2U|) . 

Markovian Quantum Control. — If we remove all con- 
straints on the control by making B full rank and letting 
Q ^ 0, Eq. (|23l simplifies to 

mopt = tiiPWu] (25) 

It turns out that the solution to this problem (the op- 
timal U) is relevant not only for optimal control, but 
also for Markovian control as introduced by Wiseman 
and Milburn Conceptually and experimentally this 
is a much simpler form of feedback, as it entails making 
the time-dependent Hamiltonian linear in the instanta- 
neous output y{t) [rather than the Kalman-filtered out- 
put, (x)^ (t)]. That is, in the context of the linear system, 

u{t) = F{t)y{t) (26) 

Note that y{t) has unbounded variation, so doing Marko- 
vian control is no less onerous than doing optimal control 
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with unbounded K{t) as occurs for Q ^ 0. With B in- 
vcrtiblc we can choose BF = —WjjC^ — F^. This makes 
Eq. I|14f) deterministic in the hmit t —^ oo: 

d(x)e = A/(x)cdt, (27) 

where M = A - WuC^C - TTC. This is the gener- 
ahzation of the optimized Markovian feedback strategy 
identified for 1-dimensional systems in Refs. |2J[. As long 
as Wu is stabihzing, M will be strictly stable [2| so that 
the solution of Eq. (|27|l will asymptote to 0. Thus from 
Eq. H23|) the cost will again be given by Eq. H25|) , and the 
optimal unravelling found as above. 

Example. — Consider a system with N — L — 1 de- 
scribed by the master equation 

Hp=^i[{qp + pq)/2,p]+V[q + ip]p, (28) 

where the output arising from the second term may be 
monitored. This could be realized in quantum optics 
as a damped cavity (harmonic oscillator in the rotating 
frame) containing an on-threshold parametric down con- 
verter 01, with p the squeezed quadrature. 

In this case we have C = (1,«), so the drift and dif- 
fusion matrices evaluate to A — diag(0, — 2), D = hi. 
Writing the conditional steady-state covariance matrix 
as 

the LMIs © and lO become 

Now say the aim of the feedback control is to produce 
a stationary state where q = p as nearly as possible. 
A suitable cost function to be minimized is ((g — p)^)^^- 
That is, ignoring any control costs, we have 

^ = (_\ "/),g-o. (31) 

In the optics case it is simple to displace the system in 
its phase space by application of a coherent driving field 
[lij i. That is, we are justified in taking B to be full 
rank. Furthermore, any quadratic cost function will be 
minimized for a pure state so we may assume that = 
1 + 0^ . Thus the m achievable by optimal or Markovian 
control is m = iY[PWu] = [(l + P^) /27 + 7/2 - P% 
constrained only byO<7< (1 — /3^)/2. The minimum 
is found numerically to be m « 1.12fi, at /3 sa 0.248 and 
7 = (1 — /3^)/2. Proceeding as described above, we find 
\/hC /2 — —T l^/h = U^/"^, with the optimal unraveling 

U=( '^"'.'^f';'^') for 0^ 0.278^. (32) 

\^ cos sm (j) sm cp J 



Optically, this unravelling corresponds to homodyne 
detection with (j) being the local oscillator phase. For 
Markovian feedback, this gives gives the drift matrix 

M = A- AWuU/h -2U^I^ Zlf^ ) . (33) 

which is strictly stable as required. 

To conclude, we have shown that even for quantum sys- 
tems that are linear (and so have a classical analogue) the 
constraints of quantum theory affect the basic structure 
of feedback control problems. In particular, we have for- 
mulated a natural question — the optimal unravelling for 
stationary LQG control problems — that has no classi- 
cal analogue. Moreover, these constraints also lead to an 
efficient algorithm to answer this question. This theory 
applies to any linearizable system with quantum-limited 
monitoring. No doubt further fundamental aspects of 
control for such systems still await discovery. 

We thank the ARC and the State of Queensland for 
support, and K. Jacobs and H. Mabuchi for discussions. 
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